Benchmark Results

java_gradle_multithreaded.csv

java_maven_serial.csv

java_simple_multithreaded.csv

java_gradle_serial.csv

java_simple_parallel.csv

java_gradle_parallel.csv

java_maven_parallel.csv

java_simple_serial.csv

java_maven_multithreaded.csv

Notes

• The results above are generated by running against seed projects in https://github.com/getgauge/gauge-benchmark
• These results are not persisted, but on merging to master the benchmark will be rerun.
• These benchmark are run in Github Actions' agents, which are virtualized. Results are not to be taken as actual benchmarks.Rather, these are indicative numbers and make sense for comparison.

See Workflow log for more details.

@sriv
OK, I found a problem in 'invokeServiceFor' not returning the correct response :(
The question now is what to do in 'executeConcept' when there is a Concept start / end response error

The question now is what to do in 'executeConcept' when there is a Concept start / end response error

I guess we can mark the concept execution result as failed like how its done here: https://github.com/getgauge/gauge/blob/master/execution/result/conceptResult.go#L53-L66

I am fairly certain that spec/scenario hooks behave similarly, i.e. mark them as failed when the hooks have failures.

@sriv
I have now an implementation that handles Concept hook result with an error.
I have verified the status and error with the following reporters:

  "Plugins": [
    "reportportal",
    "html-report",
    "xml-report"
  ],

When there is a Concept hook error, the reporters

• correctly indicate that the scenario has failed
• there is an error message and stack info
• there is presently no screenshot attached- should this be added
• there are some details that the reporters could improve

@sriv

Simulating a 'BeforeConcept' hook results in the following html-report
The error information is provided but it's not entirely clear that the concept is failing

Simulating a 'AfterConcept' hook results in the following html-report
The error information is provided but it's not entirely clear that the concept is failing

Thanks for testing this out and the feedback. I believe your feedback is around the placement of the error message?

html-report renders concepts as an expandable step with the actual steps listed under it. I guess you would like the errors in the concept hooks should be adjacent to this expandable concept?

This should be possible by changing the template for html-report. The default template lies here https://github.com/getgauge/html-report/blob/master/themes/default/views/partials.tmpl

This uses golang's stdlib templating and the concept container is here: https://github.com/getgauge/html-report/blob/master/themes/default/views/partials.tmpl#L687-L698

If you want to take a stab at this please feel free.

@sriv
Thanks for the explanation and pointing out the template functionality.
My understanding is then that there is no additional logic required in the gauge runner in order to correctly handle the concept hook status response.
In that case, can we proceed with reviewing and accepting the related gauge-proto and gauge-dotnet PRs?

Done, merged the corresponding gauge-dotnet and gauge-proto PRs

@sriv
This PR is ready ...

@PiotrNestor Thank you for contributing to gauge. Your pull request has been labeled as a release candidate 🎉🎉.

Merging this PR will trigger a release.

Please bump up the version as part of this PR.

Instructions to bump the version can found at CONTRIBUTING.md

If the CONTRIBUTING.md file does not exist or does not include instructions about bumping up the version, please looks previous commits in git history to see what changes need to be done.